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ABSTRACT 



A computer-based questionnaire was developed to help parents 
who have not received specialized training to determine whether their young 
children need further assessment for diagnosis of developmentally at-risk 
status. The computer automatically determines a starting point for a series 
of questions according to the child's chronological age. The computer also 
sets the basal point according to the child's functional level. When the 
child reaches a certain ceiling point, the computer stops asking questions, 
calculates the total numbers of "yes" and "no" answers given by the 
respondent and presents a graphical representation of the child's 
development. The questionnaire items are written in simple language and use 
pictures where possible. The validity of the instrument was examined to see 
how precisely the parent -completed questionnaire assesses children's 
development as compared to the Child Developmental Inventory, a standardized 
assessment tool. Results suggest that the parent -completed computer-based 
questionnaire is a valid tool in screening 15-month-to 36-month-old 
children's development. (Contains 32 references and 3 tables.) (Author/AEF) 
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Abstract 

A computer-based questionnaire was developed to help parents who have not 
received specialized training to determine whether their young children need further 
assessment for diagnosis of developmentally at-risk status. 

The computer will automatically determine a starting point for a series of questions 
accor^ng to the child's chronological age. The computer will also set the basal point 
according to the child's functional level. When the child reaches a certain ceiling 
pomt' tte computer will stop asking questions and will calculate the total numbers 
of yes and "no" answers given by the respondent and will present a graphical 
representation of the child's development. The questionnaire items are written in 
simple language and use pictures where possible. 

The validity of the questionnaire was examined to see how precisely the parent- 
completed computer-based questionnaire assesses children's development as 
compared to the Child Development Inventory, a standardized assessment tool. 
Correlations among the five domains of the computer-based questionnaire and the 
corresponding domains of the Child Developmental Inventory were significant and 
s rong (from r = .61 to r = .86, p < .001). The sensitivity of the computer-based 
questionnaire, as indicated by the percentage of developmentally at-risk children 
correctly identified according to the Child Developmental Inventory, was 80%. The 
speaficity of the computer-based questionnaire, as indicated by the percentage of 
children without problems correctly identified according to the Child Developmental 
Inventory, was 100%. The computer-based questionnaire had brief administration 
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time with a mean of 8.8 minutes. These results suggest that the parent-completed, 
computer-based questionnaire is a vaUd tool in screening 15-month- to 36-month-old 
children's development. 

For many years, professionals have assessed children with disabilities and determined 
goals for them based solely on their own observations (GarsheUs & McConnell, 1993). 
Parents have a rich history of observations and experiences with their children, and 
information obtained from parents represents a highly desirable source of input that 
can contribute unique and essential information to the assessment process 
(Henderson S Meisels, 1994). However, the utilization of parental information in 
assessment of young children has generally been overlooked. A major reason that 
parents have been excluded from assessment is the belief that assessments must be 
administered in a typical fashion and only by a person having required training. Even 
when assessment tools are based on naturalistic observation, parental involvement 
has been limited because of the belief that observational skills are also obtained with 
extensive training (Sheenhan, 1988). Compliance with P.L. 99-457, with such a large 
population, is difficult for at least two reasons: 1) a limited number of professionals; 
and 2) financial cost of screening. Utilizing parents to screen their children's 
development seems reasonable from an economic perspective and has parents as 
partners at the starting point of the intervention journey. However, these two 
advantages are of little use unless parents can reliably assess the child's 
performance. 

In order to help parents conduct assessment procedures in a standard way, computer 
technology can be employed, especially for the assessment of exceptional children 
(Greenwood & Rieth, 1994; Pea, 1987). Several advantages can be assumed when 
using the computer in assessment measurement. The computer can: (a) reduce the 
total assessment administration time; (b) give immediate feedback to responses; (c) 
reduce recording and computing errors, and increase scoring reUabiUty; (d) improve 
responder's motivation; and (e) provide a standard procedure of the test 
administration that can help to get information from parents reliably. 

Though recent advances in technology make it possible to improve the assessment 
procedure, only a few studies have examined the implication of computer-based 
assessment in special education. Further research is in order to examine whether 
parents can successfully complete computer-based screening questionnaires on their 
children. The validity and utility of a parent-completed, computer-based screening 
• questionnaire is examined in this study. 

Method 



The participants for this study were 46 mothers who have children aged 15 months 
to 36 months. Because this research investigated the effectiveness of a screening 
tool, participants were not excluded from the study on the basis of their child's 
developmental level. Therefore, both developmentally at-risk children's mothers and 
normal children's mothers participated in this research. Participants were recruited 
from four child care centers and three community facilities: a church, a university 
campus, and a park. Demographic information collected from the 46 mothers at the 
time of the Computer-Based Developmental Questionnaire completion is presented in 
Table 1. 






nj 




3 



119 



Table 1. Demographic Information of the Participants 



Demographic Variable 


Frequency (%) 


Child Gender 


Male 


31 


(67.4) 


Female 


15 


(32.6) 


Level of Maternal Education 


Less than 12 years 


3 


(6.5) 


12 years 


5 


(10.9) 


13 to 15 years 


18 


(39.1) 


16 or more years 


18 


(39.1) 


Not answered 


2 


(4.3) 


Annual Family Income 


Less than $10,000 


14 


(30.4) 


$10,001 to $20,000 


5 


(10.9) 


$20,001 to $50,000 


13 


(28.3) 


Over $50,000 


12 


(26.1) 


Not answered 


2 


(4.3) 



Participants rated their children with the Computer-Based Developmental 
Questionnaire (CBQ) and the Child Development Inventory (CDI; Ireton, 1992). The 
CDI is a standardized instrument used to measure the development of children 15 
months to 6 years of age. It has a 270-item inventory that describes eight 
developmental scales of young children in the areas of social, self-help, gross motor, 
fine motor, expressive language, language comprehension, letters, and numbers. The 
CBQ is designed for screening 9-month- to 36-month-old children to identify whether 
they need further developmental assessment. The questionnaire screened in five 
areas: adaptive-psychosocial, cognitive, speech-language, fine motor, and gross 
motor. To determine the items asked in the CBQ, the researcher reviewed child 
development literature and standardized developmental screening scales 
(Frankenburg, Fandal, & Thornton, 1987; Ireton & Thwing, 1974; Knobloch, Stevens, 
& Malone, 1980; Lowrey, 1986; Princeton Center for Infancy and Early Childhood, 
1977). From this body of literature, the researcher identified a total of 190 items 
that could be answered "yes" or "no" by parents. Authorware, an application 
software, and a Power Macintosh 7100/80 computer were used to transform 
questions taken from literature into the electronic questionnaire. 

The 46 participants were randomly divided into two subgroups by the order of test 
completion. The mothers completed both the CBQ and the CDI. One subgroup took 
the CBQ first and the CDI later; the other subgroup took the CDI first and the CBQ 
later. The groups were divided in this way to control any possible effects of the 
participants' prior completion of one test on the other test. When completing the 
CBQ, participants answered questions in five developmental domains. Each response 
was made by indicating either "yes" or "no." Ceiling points in each of the five areas 
were set based on the answers given by the respondent. A ceiling point in a domain 
was a question answered with "yes" followed by two questions answered with "no." 





4 



This "yes" question had a point-value associated with it, which determined the raw 
score for that particular developmental area. Each child thus had raw scores in the 
five domains as well as one average score that was a general developmental score. 
The general developmental score and the five domain scores from the CBQ, in turn, 
were conelated with conesponding scores from the GDI. The data analysis was 
calculated using computational procedures of SPSS 6.0 (SPSS, Inc., 1993). 



Results 

Validity of the CBQ 

A conelation coefficient is one way to see the degree of relationship between a 
screening measure and a standardized test to verify concunent validity. Pearson 
product moment conelation coefficients were calculated between the two tests. In 
order to control for the effects of child's age on the various scores of the CBQ and 
the GDI, partial conelation analysis was used. The partial conelation coefficient 
provides a measure of Unear association between two variables while adjusting for 
the linear effects of one or more additional variables (Norusis & SPSS, Inc., 1993). 
Table 2 contains the conelation coefficients between the major CBQ component 
scores and the GDI scores after the effects of child's age are removed. As can be seen 
in Table 2, the conelations between the CBQ and the GDI are quite high and 
significant. The general development score of the GDI that is considered to be most 
related to learning in school significantly conelates with all the subscales of the CBQ 
(from r = .59 to r = .85, p < .001). A high conelation was found (r = .83) on the 
general development scores of the two tests. The conelations among five domains of 
the CBQ and the conesponding components of the GDI are presented in bold in Table 
2 (from r = .61 to .86, p < .001). Those five conelation coefficients show that those 
are more strongly conelated with conesponding domains than with any other 
domains. Cognitive domain of the CBQ does not have an exactly conesponding 
domain in the GDI, however it is strongly conelated with the general development 
and the social domain of the GDI (r = .61 and r = .68 respectively). Overall, the 
conelations between the CBQ and the GDI offer strong support for the concunent 
vaUdity of the CBQ. These strong conelations between the two measures suggest that 
they may be measuring the same constructs. 



Table 2. Correlations between the CBQ and the CDI 



Test Domains - 


BAD 


BCO 


BSL 


BFM 


BGM BGD BDQ 


CBQ 












Adaptive (BAD) 












Cognitive (BCO) 


.64*** 










Speech-Language (BSL) 


.78*** 


.63*** 








Fine Motor (BFM) 


.56*** 


. .50*** 


.63*** 






Gross Motor (BGM) 


.65*** 


.50*** 


.59*** 


.49** 




General Development (BGD)+ 


.89*** 


.78*** 


.93*** 


.74*** 


.78*** 


Developmental Quotient (BDQ)+ 


.88*** 


.78*** 


.83*** 


.77*** 


.79*** .98*** 



(table continues) 
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Table 2. (continued) 



Test Domains 



BAD BCD BSL BFM BGM BGD BDQ 



GDI 



Gross Motor 
Fine Motor 
Expressive Language 
Language Comprehensive 
General Development + 



Social 
Self Help 




Note: Correlation coefficients are Pearson product-moment rs controlled for child's age. 

Convergent validity coefficients are presented in bold. 

+ These components are not independent domain because they are calculated from the other domains. 
**p <.01, two-tailed test 
***p <.001, two-tailed test 



Validity of the CBQ was also examined by comparing the classification of the children 
by the CBQ and the GDI. Each child weis screened for being normal or developmentally 
at-risk with both the CBQ and the GDI. The percentages of agreement between the 
two tests were then calculated. No universally accepted definition of developmental 
delay exists. However, definitions usually require that the child exhibit a 20 to 30% 
delay in functioning when compared to his or her peers (Bayley, 1993; Eisert, 

Spector, Shankaran, Faigenbaum, & Szego, 1980; Ireton, 1992). In this study, a 
child's development on the CBQ and the GDI was classified as developmentally at-risk 
if the child's scores fell below the 30% below age cutoff on any domain. The 30% 
below age cutoff of the GDI was reported as equivalent to two standard deviations 
below the mean (Ireton). 

The results of the CBQ classified 34 children as normal and 12 children as at-risk 
(Table 3). Meanwhile, 31 children were classified as normal and 15 children were 
classified as at-risk on the GDI when the 30% below cutoff was applied. The 
sensitivity of a testing instrument is the percentage of developmentally at-risk 
children correctly identified according to a standardized test. Twelve of the 15 who 
were identified as at-risk by the GDI were also identified as at-risk by the CBQ, so 
that the CBQ's classifications were 80% (12/15 x 100) sensitive in the identification 
of at-risk children. The specificity of a testing instrument is the percentage of 
children without problems correctly identified. All 31 children who were classified as 
normal by the GDI were also identified as normal on the CBQ. All classifications of 
children as normal by the CBQ agreed with classifications of children by the GDI. 
Therefore, the CBQ was 100% (31/31 x 100) specific in identifying children with 
normal development. In summary, the classification of children's development as 
normal or at-risk by the CBQ agreed highly vdth the classifications of children by the 
standardized test (GDI) which verifies the validity of the CBQ. Overscreening and 
underscreening are two potential errors that can occur during any kind of screening. 
Overscreening refers to the percentage of children labeled as at-risk by the screening 
test who are found to be normal by the criterion measure. Underscreening refers to 
the percentage of children not detected by the screening test who are found to be 
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at-risk oi\ the criterion test (Wolery, 1989). All 12 children who were classified as at- 
risk on the CBQ were also identified as at-risk on the GDI. No overscreening case was 
found on the CBQ. However, there were three children who were classified as normal 
on the CBQ who were found to be at-risk on the GDI. The CBQ's underscreening was 
20% (3/15 X 100). 



Table 3. Agreement between the two tests 

GDI 



Normal At-Risk 



CBQ 



Normal 



At-Risk 



31 


3 


0 


12 



Sensitivity; 

Specificity; 

Overscreening 

Underscreening 



12/15 X 100 = 80% 
31/31 X 100 = 1007o 
0/31 X 100 = 07o 
3/15 X 100 = 207o 



Utility of the CBQ 

The numbers of questions mothers answered varied from mother to mother because 
the computer automatically determines a starting point for a series of questions 
according to the child's chronological age and an ending point to the level of a 
child's development. The number of questions answered by mothers ranged from 26 
to 90 items, with a mean of 47 items {SD = 12.70). Time for completion of the CBQ by 
the mothers was measured by the computer. It included participants' answering the 
demographic pages, answering the questionnaire, calculating developmental scores, 
presenting developmental scores on a profile, and saving the data in a file to the 
computer. Completion time ranged from 5 minutes to 16 minutes with a mean of 8.8 
minutes {SD = 2.77) and a mode of 7 minutes. The GDI required that parents answer 
270 questions. It takes about 30 minutes for the parent to complete the test and 5 
.. minutes to score the results. In contrast, on the CBQ, parents answered about 50 
questions and it took about 9 minutes to complete and to score. 



Discussion 

Correlation Data, Sensitivity, and Specificity of the CBQ 

In earlier studies, significant positive conelations were found between parental 
assessment of children and a professionally administered standardized assessment 
(Gradel, Thompson, & Sheehan, 1981; Sexton, Hall, & Thomas, 1984). For example, 
Sonnander (1987) reported results for a parent-completed screening of 18-month-old 
children in which a high conelation was found (r = .87) between standardized test 
scores and parental assessment scores. Correlations between the five domains of the 
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CBQ and the corresponding domains of the CDI ranged from .61 to .87. The high 
correlations suggest that the CBQ and the CDI are relatively consistent in their 
ratings of the children's development. Glascoe and Byrne (1993) studied the validity 
of three developmental screening tests. Those tests were the Denver-II (Frankenburg, 
Dodds, Archer, Bresnick, Maschka, Edelman, 8i Shapiro, 1990), the Battelle 
Developmental Inventory Screening Test (BDI; Newborg, Stock, Wnek, Guidabaldi, & 
Svinicki, 1984), and the Academic Scale of the Developmental Profile-n (DP-II; 

Alpem, Boll, & Shearer, 1986). Glascoe and Byrne suggested that a sensitivity of at 
least 80% was preferred and a specificity of at least 90% was preferred. The 
sensitivity of Denver-n was adequate (83%), but the specificity of Denver-n was low 
(46%). In other words, the Denver-n is good at detecting children with difficulties, 
but it produces a high overreferral rate. This overreferral results in needless parental 
anxiety and expense, and wastes limited diagnostic resources. DP-H showed adequate 
specificity (86%), but the sensitivity of the scale was found to be very low (22%). 

The BDI was more accurate than the other screening tests. The sensitivity and 
specificity of the BDI were both 72%. Bricker and Squires (1989a, 1989b) examined 
the validity of a parent-completed screening system called the Infeint Monitoring 
Questionnaires (IMQ). The specificity of the questionnaires was high (over 90%), but 
the sensitivity was low (63%). This study found that sensitivity of the CBQ was 80% 
and specificity was 100%. These results suggest that the CBQ is very sensitive to 
identify developmentally at-risk children correctly and very specific to exclude those 
children who are normal. However, there is some evidence that the CBQ underscreens 
children who need further assessments (underscreening = 20%, Table 3). 

Completion Time and Cost of the CBQ 

Numerous instruments exist which can be more or less useful in screening 
developmentally at-risk children. However, the majority of the current screening 
measures are lengthy and have time consuming procedures to interpret the results 
(Glascoe & Byrne, 1993; Glascoe, Martin, & Humphrey, 1990). Screening measures 
usually take 20 to 30 minutes to administer and 5 to 10 minutes to score and to 
interpret. The Denver-n, which is administered by professionals, is reported to take 
20 minutes to complete, plus the time involved in scoring the results (Kenny, Hebei, 
Seston, & Fox, 1987). The BDI screening test takes 10 to 15 minutes to administer for 
children under 3; for children between 3 and 5, it takes 20 to 30 minutes (Newborg, 
Stock, Wnek, Guidabaldi, & Svinicki, 1988). The Developmental Profile II usually 
requires approximately 20 to 40 minutes to administer and score (Alpem, Boll, & 
Shealer, 1986). The CDI used in this study as a criterion measure usually takes 30 
minutes of parents' time to complete and 5 minutes to score. In contrast, the CBQ 
took about 7 to 10 minutes (mean = 8.8, SD = 2.8) by the mothers who did not have 
a specialized training. The time included completion of demographic screens, 
completion of questionnaire, scoring, and recording. Because demographic screens 
were made only for use in this study, without these pages, a reduced time will be 
expected for a general use of the CBQ. The number of questions answered varied by 
mothers. It ranged from 26 to 90 items with a mean of 47 items (SD = 12.70). These 
results occurred because mothers were given different numbers of questions 
according to their children's age eind development. This computer system omits 
unnecessary questions beyond the level of the child's development. 
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Cost clearly plays a role in the adoption of computer-based assessment. Although 
development and capital cost may be higher for technology-based assessment, it may 
yield substantial savings because development costs are one time expenses (Garland, 
1995). The cost of implementing GDI for 46 children was about $90 including a 
manual, questionnaires, answering sheet, and scoring sheet. The cost of 
implementing the GDI will be multiplied by the numbers of children. This is a case of 
a parent-administered test so it is relatively inexpensive. However, if a professional 
needed to administer a screening measure, that system would be more costly. The 
CBQ eliminated professional time and paperwork, so once it is developed, there is no 
further cost. 

Because of the small sample size, this study could not investigate relationships 
among child development, parents' ethnic background, and parents' socioeconomic 
status. In addition, as the GBQ was written in English so the use of the GBQ with 
groups whose primary language is not English would not be appropriate. 

Implications for Future Research 

The conelations among the children's various developmental scores on the GBQ and 
the GDI were significant and strong. The percentages of agreement between the two 
tests in identifying children as normal or as developmentally at-risk were also high. 
However, it should be noted that the GBQ is not a diagnostic test. It is only a 
screening test; therefore, further full scale assessment is needed to confirrn the 
developmental condition of the child. It is important to recognize that a single 
screening measure of a child with substantial risk factors cannot be used either to 
confirm or rule out developmental delay (Katz, 1989). A low scoring child is only at- 
risk, and maybe only temporarily so (Sonnander, 1987). Therefore, the predictive 
validity, the extent to which the screening test agrees with children's performance 
on outcome measures later in time, should be investigated. The specificity of the GBQ 
was 100%, but some underscreening of the GBQ (sensitivity = 80%; underscreening = 
20%) was found. Because establishing criterion-related validity is an ongoing process 
(Allen & Yen, 1979), further studies are necessary to study the validity of the GBQ 
with other standardized measures. 
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